must be strong and evident commitment by the local agency to the used an experimental design and clear understandings with researchers i;: assure that they have sufficient authority to see that the program design;; implemented with integrity. It may be easier to negotiate with a provide of services than a governmental agency subject to bureaucratic regulation and local political pressures.
A disadvantage of this approach is that the research may not fit neat!; into the development of a particular program model. But given the limitation on opportunities to implement experimental interventions, particularly with adjudicated offenders, it is a useful way to increase those opportunities for field experiments and, through a variety of inducement) that involve the staff of state and local agencies in the research process,!: increase the chances of maintaining the integrity of the experimental design.
When a promising program appears ready for wider testing, field tests in one or more sites must be arranged. At NIJ the Office of Development, Testing and Dissemination develops a program model, devises a field test design for implementation in three sites, and plans a separate evaluation of the program. This policy is an improvement over the previous strategy of moving right into a national action or demonstration program tested in as many as a dozen sites because it permits more carefully controlled research. When evaluators are presented with a large number of very diverse projects sharing only the same label, there is no overall "program,1 so that they must "evaluate" what is virtually unevaluable.
An essential element in a field test of a program model, even at as few as three sites, is the coordinated development and integration of the program and evaluation designs. The goals of a program and the assumptions on which it rests must be clear enough to permit the evaluation of the program. In addition, the program elements and procedures must be consis' tent with, grow from, and test those assumptions. The evaluation design grows from the program model that serves as the basis for project site activities. In a multisite program evaluation, it is necessary to make sure that the program model being tested at each of the sites actually is the same or to document the ways in which the models differ. A multisite program must rest on and test theoretically related propositions so that differing program elements become systematically designed variations on a theme.
A critical issue in designing a field test that permits the acquisition of knowledge about the model's effectiveness is the question of the timing and execution of the evaluation and the project site activities. Often evaluations of criminal justice programs are devised as an afterthought, following the development and implementation of a new program. Suchheoretically grounded effort that is evaluable and that the intervention and evaluation are designed conjointly.
